User Simulation in Dialogue Systems Using Inverse Reinforcement Learning
نویسندگان
چکیده
Spoken Dialogue Systems (SDS) are man-machine interfaces which use natural language as the medium of interaction. Dialogue corpora collection for the purpose of training and evaluating dialogue systems is an expensive process. User simulators aim at simulating human users in order to generate synthetic data. Existing methods for user simulation mainly focus on generating data with the same statistical consistency as in some reference dialogue corpus. This paper outlines a novel approach for user simulation based on Inverse Reinforcement Learning (IRL). The task of building the user simulator is perceived as a task of imitation learning.
منابع مشابه
Behavior Specific User Simulation in Spoken Dialogue Systems
Spoken dialogue systems provide an opportunity for man machine interaction using spoken language as the medium of interaction. In recent years reinforcement learning-based dialogue policy optimization has evolved to be state of the art. In order to cope with the data requirement for policy optimization and also to evaluate dialogue policies user simulators are introduced. Almost all existing da...
متن کاملA Two-Tier User Simulation Model for Reinforcement Learning of Adaptive Referring Expression Generation Policies
We present a new two-tier user simulation model for learning adaptive referring expression generation (REG) policies for spoken dialogue systems using reinforcement learning. Current user simulation models that are used for dialogue policy learning do not simulate users with different levels of domain expertise and are not responsive to referring expressions used by the system. The twotier mode...
متن کاملReinforcement Learning of Cooperative Persuasive Dialogue Policies using Framing
In this paper, we apply reinforcement learning for automatically learning cooperative persuasive dialogue system policies using framing, the use of emotionally charged statements common in persuasive dialogue between humans. In order to apply reinforcement learning, we describe a method to construct user simulators and reward functions specifically tailored to persuasive dialogue based on a cor...
متن کاملA survey of statistical user simulation techniques for reinforcement-learning of dialogue management strategies
Within the broad field of spoken dialogue systems, the application of machine-learning approaches to dialogue management strategy design is a rapidly growing research area. The main motivation is the hope of building systems that learn through trial-and-error interaction what constitutes a good dialogue strategy. Training of such systems could in theory be done using human users or using corpor...
متن کاملHybrid Reinforcement/Supervised Learning for Dialogue Policies from COMMUNICATOR data
We propose a method for learning dialogue management policies from a fixed dataset. The method is designed for use with “Information State Update” (ISU)-based dialogue systems, which represent the state of a dialogue as a large set of features, resulting in a very large state space and a very large policy space. To address the problem that any fixed dataset will only provide information about s...
متن کامل